Model Validation Methodology : From Validation Experiments to Systems Level Application

Abstract— Our increased dependence on computer models leads to the natural question – How do we increase the rigor in validating models against experimental data? Models have traditionally been tested against experimental data through simple comparisons such as x-y plots, scatter plots, or contour plots. While such qualitative comparisons are appropriate for model building, the use of such comparisons for model validation naturally leads to the questions – When is the agreement between experimental measurements and model predictions sufficient, and how should we quantify this agreement? Unfortunately, defining rigorous metrics for such comparisons is difficult since there are uncertainties in the validation experiment measurements and in the model parameters. Because of these uncertainties, we expect there to be differences between experimental observations and model predictions, even for perfect models. In addition, when models predict multivariate data (time histories or spatial distributions for example), the differences between the experimental observations and model predictions can be highly correlated. Furthermore, we often measure one quantity from a validation experiment, but desire to predict another quantity for the target application of our model. Finally, complex multi physics models often require a suite of validation experiments to test the model over the range of parameters and physics addressed by the anticipated target application. Here we present an approach to the development of rigorous model validation methodology that accounts for measurement and model parameter uncertainty. Specifically, we present methodology, based on first order sensitivity analysis, to 1) evaluate whether the validation experiments “cover” the physics of the target application, 2) combine the validation data from a suite of experiments to best represent the decision variables for the target application, 3) evaluate the uncertainty associated with the combined data, and 4) define a validation metric based on this combination of data that accounts for uncertainty in the experimental measurements and the model parameters. We present examples of the methodology based on thermal diffusion.